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Abstract — As many sensor network applications require de- 
ployment in remote and hard-to-reach areas, it is critical to 
ensure that such networks are capable of operating unattended 
for long durations. Consequently, the concept of using nodes with 
energy replenishment capabilities has been gaining popularity. 
However, new techniques and protocols must be developed to 
maximize the performance of sensor networks with energy 
replenishment. Here, we analyze limits of the performance of 
sensor nodes with limited energy, being replenished at a variable 
rate. We provide a simple localized energy management scheme 
that achieves a performance close to that with an unlimited 
energy source, and at the same time keeps the probability of 
complete battery discharge low. Based on the insights developed, 
we address the problem of energy management for energy- 
replenishing nodes with finite battery and finite data buffer 
capacities. To this end, we give an energy management scheme 
that achieves the optimal utility asymptotically while keeping 
both the battery discharge and data loss probabilities low. 

I. Introduction 

Advances in wireless networking combined with data acqui- 
sition have enabled us to remotely sense our environment [1], 
121 ■ As these applications may require deployment in hard- 
to-reach areas, it is critical to ensure that such networks are 
capable of operating with full autonomy for long durations. 
The lack of a continuous power source in most scenarios and 
the limited lifetime of batteries have hindered the deployment 
of such networks. However, developments in renewable energy 
sources |[3l-|l9l suggest that it is feasible for sensor networks 
to operate unattended for extended periods. These renewable 
sources of energy typically provide energy replenishment at a 
rate that could be variable and dependent on the surround- 
ings. Examples include, self-powered sensors that rely on 
harvesting strain and vibration energies from their working 
environment |4i, as well as sensors with solar cells JSj-lTl. 

In this paper, we analyze the limits of the performance of 
networks comprised of sensor nodes with limited energy, being 
replenished at a variable rate. We provide a simple localized 
energy management scheme that achieves a performance, close 
to the optimal scheme that has access to an unlimited energy 
reservoir Indeed, we show that, if the performance can be 
measured by a general utility function of the energy, under 
mild assumptions on the replenishment process, it is possible 
to observe a polynomial decay for the probability of complete 
battery discharge, and at the same time achieve a Q ^ 
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convergence to the optimal achievable utilitjQ. Here M is the 
total capacity of the energy source. Based on the insights 
developed, we address the problem of energy management in 
the presence of a finite data buffer We modify our basic energy 
management scheme to achieve a Q ^ convergence to 

the maximum utility achievable by a scheme that has access to 
an infinite data and energy buffers. Here K is the data buffer 
size. In addition, this scheme achieves an exponential decay 
with AI for the battery discharge probability and a polynomial 
decay with K for the data loss probability. To evaluate these 
decay rates, the main tools we use are the large deviations 
theory and diffusion approximations. 

The added dimension of renewable energy makes the prob- 
lem of energy management in sensor networks substantially 
different from its non-replenishment counterpart. For nodes 
with replenishment, conservative energy expenditure may lead 
to missed recharging opportunities due to battery capacity 
limitations. On the other hand, aggressive usage of energy 
may cause battery outages that leads to lack of coverage or 
connectivity for certain time periods. Thus, new techniques 
must be developed to balance these seemingly contradictory 
goals to maximize performance. Here, our main goal will be 
to identify the performance limits of sensor nodes with energy 
replenishment and provide guideUnes to approach these limits. 

Many fundamental wireless communication and networking 
problems can be stated as utility maximization problems, 
subject to energy constraints. The utility function can be the 
throughput (e.g., in energy efficient routing), the probabiUty 
of detection of an intruder (e.g., in coverage) or the network 
lifetime (e.g., in sleep-wake scheduling) or the achievable 
rate of reliable transmission in basic wireless communication. 
These problems have been mainly addressed for stations with 
unlimited and/or non-replenishing energy sources. Here, we 
address the problem of maximizing a utility function of the 
data transmission rate in the presence of energy replenishment. 
The solution of the optimization problem requires stochastic 
optimization techniques involving high computational over- 
heads that might be unsuitable for sensor nodes. Consequently, 
we will focus our attention on simple localized solutions that 
achieve near-optimal or asymptotically optimal performance. 
We use tools from large deviations theory and diffusion 
approximation to find closed-form expressions for the data 
loss and the battery discharge probabilities. These techniques 
allow us to analyze our schemes under mild assumptions on 
the battery charging and data arrival processes. 

'The following notations will be used to compare rates of convergence: 
a„ = 0(6„) if a„ goes to zero at least as fast as 6„; a„ = o(6„) if a„ 
goes to zero strictly faster than 6„; a,i = 0(bn) if fin and 6„ go to zero at 
the same rate; a„ = f!(fen) if goes to zero no faster than 6„. 
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Fig. 1. Energy source witli a replenishment rate r{t). 



There have been recent works that have studied differ- 
ent problems in networks with energy replenishment. Kar, 
et. al., ITOl have proposed an activation scheme for recharge- 
able sensors that maximizes the network-level utility of sens- 
ing networks. The utility function in ifTOl depends on the 
number of active sensors. Gatzianas, et. al., ifTTl use back 
pressure policies to maximize the network flow of information 
in networks with energy replenishment. While ifTol and ifTTI 
look at the total system utility, we will focus on the an- 
alyzing node-level performance leading to localized energy 
management schemes. Liu, et. al., lfT2l have derived a battery 
control scheme similar to the one described in this work. In 
addition to providing stronger convergence results than the 
one in [12 J with sole battery control, we also consider the 
effect of a finite data buffer in this paper. Kansal, et. al., lITSi 
introduce the concept of energy neutral operation, wherein the 
energy consumed by a node is less than or equal to the energy 
harvested. Vigorito, et. al., llT4l extend the idea of energy 
neutral operation to propose an algorithm that attempts to 
keep the battery state close to a fixed level and at the same 
time stabilizes the duty cycle in order to maximize system 
performance. Sharma, et. al., ifTSl have proposed a throughput 
optimal energy management scheme for energy harvesting 
nodes. However, these works ITSl - llTSIl do not contain an 
analytical evaluation of the battery discharge or the data loss 
probabilities for their energy management schemes. 

The outline of this paper is as follows. We first state the 
general form of the utility maximization problem in Section Ull 
and show ways to achieve the maximum achievable utility with 
replenishing sources. In Section|lII]we add a finite buffer to the 
problem and study energy management schemes that achieve 
optimal utility asymptotically while keeping the probabilities 
of battery discharge and data loss low. We numerically eval- 
uate the performance of our energy management schemes in 
Section |IV] We wrap up with conclusions in Section |V] 

II. Achieving Maximum Utility With a 
Finite-Battery Constraint 

A. System Model and Problem Statement 

Fig. [T] shows the energy source (or the battery) of a node. 
The total capacity of this battery is M units of energy. We 
denote the total available energy in the battery as B{t), where 
t is the discrete time index. The battery replenishes at a rate 
r{t). The replenishment process {r{t),t > 1} is assumed 
to be an ergodic stochastic process with a long term mean 
limi-^oo 7 St=i ' (0 = /i. A power allocation policy S draws 
energy from this battery at a rate e^{t) to achieve certain tasks. 
The success of the node in achieving these tasks is measured 
in terms of a concave and non-decreasing utility functior|| 

^Note that, in many practical scenarios, it is reasonable to assume that 
the utility function is non-decreasing and concave, since there is diminishing 
returns for increasing power. 



U{e'^{t)) of the consumed energy e'^(t). We define the time 
average utility. 



U^{r) = -j2uie''{t)). 



(1) 



t=i 



We consider the optimization problem in which a node 
tries to maximize its long-term average utility, — 
limT-_j.oo U^{t), subject to battery constraints: 



max 

{eS(t), t>l} 



(2) 



subject to B{t) = max {O, min{Af, B(t - 1) 
+ r{t)-e^{t-l)}} 
and e^{t) < B{t). 

One approach to solving this optimization problem is by using 
Markov decision process (MDP) techniques. Since solving 
MDPs is computationally intensive, these methods may not 
be suitable for computationally-limited sensor nodes. Con- 
sequently, we seek schemes that are easy to implement and 
yet achieve close to optimal performance. The next lemma 
gives an upper bound for the asymptotic time-average utility 
achieved over all ergodic energy management policies. 

Lemma 1. Let U* be the solution to Problem (|2]). Then, 

u*" < U{n). 

The proof of this lemma, given in Appendix |A1 uses 
Jensen's inequality and conservation of energy arguments. 
Lemma [T] tells us that for any ergodic energy management 
scheme S, < U{n). With an unlimited energy reservoir 
(i.e., M = oo) and average energy replenishment rate /i, if one 
uses e^{t) ~ 11 for all t > 1, this upper bound can be achieved. 
However, if M < oo, achieving = U (/i) using this simple 
scheme is not possible. Indeed, due to finite energy storage 
and variability in r{t), B{t) will occasionally get discharged 
completely. At such instances, e"^ [t) has to be set to 0, which 
will reduce the time-average utility. 

The question we answer next is, "how close can the average 
utility get to the upper bound U{ij) asymptotically, as 
M — > oo, while keeping the long-term battery discharge rate 
low?" 

B. An Asymptotically Optimal Power Allocation Scheme 

In this section we show that there is a trade-off between 
achieving maximum utility and keeping the discharge rate low. 
First, we make some weak assumptions on the replenishment 
process r(t), which we will be using throughout this paper In 
particular, we assume that the asymptotic semi-invariant log 
moment generating function. 



Kr{s) 



lim — log 



cxp 



(3) 



of r{t) exists for s E (— oo, Smax), for some Smax > 
0. We also assume that the asymptotic variance = 
limi-^oo 7var (^^^j^ 7-(t)) of r(t) exists. Note that, in prac- 
tice, the recharging process is not necessarily stationary. While 
this assumption does allow the possibility that the statistics of 
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Fig. 2. With scheme B, utility alternates between J7+ and U~ 

r{t) has variations (e.g., due to clouds and the solar power 
at different times of the day), it rules out the possibility of 
long-range dependencies in r{t). 

From the discussion in previous section, we can infer that 
by choosing a battery drift that goes to zero with increasing 
battery size, one might achieve a long-term average utility that 
is close to U (fi) as AI increases. However, smaller drift away 
from the empty battery state implies a more frequent occur- 
rence of the complete battery discharge event. In the following 
theorem, we quantify this tradeoff between the achievable 
utility and the battery discharge rate, asymptotically in the 
large battery regime. In this regime, the battery size M is large 
enough for the variations in r{t) to average out nicely over the 
time scale that B{t) changes significantly. Consequently, we 
now define the long-term battery discharge rate as the probabil- 
ity of discharge, i.e., Pdischaige(M) = limr^oo 7 Lt^i^^^ 
where the indicator variable I(f (t) = 1 if B{t) =0 and is 
identical to otherwise. 

Next, we show that one can achieve a battery discharge 
probability that exhibits a polynomial decay with the battery 
size, and at the same time achieves a utility that approaches 
the maximum achievable utility as (log Ajf)^/i\/^. 



concave and non- 

d^Uje) 
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Theorem 1. Consider any continous, 
decreasing utility function U{e(t)) such that — gjr 
for all e > 0. Given any /3 > 2, there exists a power 
allocation scheme B such that the associated battery discharge 
^(M) = Q{M-f^) and U{fi) - ~ 



probability 

og M " 
M 



We give a brief sketch of the proof, details of which can 
be found in Appendix |B] Our proof is constructive as we 
show a strategy that achieves the asymptotic convergence rates 
given in Theorem [T] Our scheme is motivated by the buffer 
control strategy introduced in |i,16,,| to achieve the near-optimal 
distortion for variable rate lossy compression. Consider the 
allocation scheme B in which 




B{t) < M/2 
B(t) > M/2 ' 



(4) 



for some d'^ > 0. As shown in Fig. |2] the instantaneous utility 
associated with Scheme B alternates between and 
depending on the battery state. By choosing Sf = ^'^ 



M 



for some /? > 2, we show that long-term maximum utility 
U{fi) can be achieved asymptotically while achieving decay, 
as a polynomial of arbitrarily high order, for the battery 



discharge probability. We note that the while the order of the 
polynomial decay (3 can be made arbitrarily large, it comes at 
the expense of slower convergence (by some constant factor) 
to the maximum utility function. 

Here, we illustrated that with a simple scheme, it is possible 
to achieve desirable scaling laws for the performance of a 
given task, under the assumption that the asymptotic moment 
generating function of the replenishment process exists. To 
illustrate the theorem we consider a specific example. 

Example 1 Capacity of an AWGN Channel 

We study the basic limits of point to point communication 
with finite but replenishing energy sources. For simplicity, we 
consider the additive white Gaussian noise (AWGN) channel. 
At time t, the transmitter transmits a complex valued block 
(vector of symbols) X(<) and the receiver receives Y(t). We 
have, 

Y{t) = hX{t)+W{t), (5) 

where the channel gain h is a complex constant and W(t) 
is additive white (complex) Gaussian noise with two sided 
power spectral density No/2. We define the channel SNR as 
7 = E /Nq. The maximum amount of data that could be 
reliably communicated fTT^ over this channel with an amount 
of energy e(t) at time t is. 



C(e(<)) = log2 (1 + e{t}j) bits/channel use, 



(6) 



assuming the block size is long enough so that sufficient 
averaging of additive noise is possible. Thus, the rate at which 
reliable communication can be achieved is a concave non- 
decreasing function of the transmit power and it can be viewed 
as our utility function. Consequently, using a constant power 
/i, the maximum utility of C* = C{fj.) can be achieved, which 
is the famous AWGN channel capacity result. 

Now, we generalize the AWGN capacity result to the case 
with finite energy sources. Suppose that we want to transmit 
the maximum amount of data over the AWGN channel, using 
a battery of energy capacity M and a replenishment rate r{t). 
We assume that each time slot is long enough for sufficiently 
long code blocks to be formed. We substitute U{-) with C(-) 
in Eq. ([T]) to get the relevant optimization problem. With an 
unlimited energy source (AI = 00) of limited average power 
yU, the maximum achievable long term average rate, i.e., the 
channel capacity is C(/.i) = log2(l + ^7) bits/channel use. 
By using the energy management scheme B given in Eq. (01), 
an average rate can be achieved such that C{p) — = 
(^ ^^°fiP ^ while the battery discharge probability follows 
Pd1scharge(A'/) = Q{M-^) for somc /3 > 2. 

C. Basic Limits of Power Allocation Schemes 

To understand the strength of Theorem [H we note that it is 
not trivial to achieve decaying discharge probability and max- 
imum utility with increasing battery size. In fact, an erg odicl 
energy management scheme cannot achieve exponential decay 
in discharge probability and convergence (even asymptotically) 

^An ergodic energy management scheme [t] is the one that satisfies 

linv^ooiELie^{*)=E[e^(i)] 
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to the maximum average utility function simultaneously. We 
formalize this statement in the following theorem. 

Theorem 2. Consider any continous, concave and non- 
decreasing utility function U{-). If an ergodic energy manage- 
ment scheme S has a discharge probability P%schargei-^) ~ 
0(exp(— acM)) for some constant Uc > 0, then the time 
average utility, IJ'^ , for Scheme S satisfies U{^)~U^ ~ ^(1)- 

The proof of this theorem is provided in Appendix |C] and 
it is similar to that of Theorem [T] We apply large deviations 
technique to the net drift of the battery process to find the 
decay rate of pfischai-ee(^^) ^i'^^ M. Jensen's inequality is then 
used to lower bound the difference between U{^) and tj^ . 

So far, we have shown how to maximize a concave non- 
decreasing utility function subject to battery constraints. At 
every point in time, one should choose a power level as close to 
the replenishment rate as the battery constraints allow and this 
way one can asymptotically achieve a performance very close 
to that with unlimited energy sources. The main limitation 
of this approach is that it may not be feasible for some 
applications in practice. For instance in many sensor network 
applications, data is stored in finite buffers for transmission. 
Since scheme B does not adapt to the buffer state, this may 
lead to data losses. To overcome these limitations, in the next 
section, we investigate energy management schemes with finite 
buffer and battery constraints. 



III. Achieving Maximum Utility With Finite 
Buffer and Battery Constraints 

A. System Model and Problem Statement 

In this section, we extend the problem introduced in Sec- 
tion In] to the case when data packets arrive at a node and 
are kept in a finite buffer before transmission. Hence, the 
task is to transmit packets arriving at the data buffer without 
dropping them due to exceeding the buffer capacity. We define 
Q{t) as the data queue state at time t, and the data buffer 
size is K < oo. The data arrival process a{t), represents 
the amount of data (in bits) arriving at the data buffer in the 
time slot t. The process {a{t),t > 1} is an ergodic process 
independent of the energy replenishment process {r{t),t > 1} 
and E [a(T)] = A. We assume that the process a{t) has a finite 
asymptotic variance = limi— yoo 7var (^JLj^ a(i)). The 
energy replenishment model is the same as used previously. 
We use C( ) as given in Eq.® as the rate-power function 
for the wireless channel and assume that data is served at 
that rate as a function of the consumed energy e(t) at time 
t. We also assume that A < C{fi), which is a necessary 
condition for system stability ifTsl . Without this condition, 
there exists no joint energy and data buffer control policy that 
can simultaneously keep the long-term battery discharge and 
data loss rates arbitrarily low. 

The objective of an efficient energy management scheme 
in this case is to maximize the average utility function of the 
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Fig. 3. Possible drift directions for (Q(t), B{t)) for an AWGN channel of 
channel SNR dB. Here, at time t, r{t) = 0, a{t) = 0. 

data transmitted subject to battery and data buffer constraints: 

max lim -^^ UoiCieit))) (7) 

e(t), t>l T-i-oo T 

subject to B{t) = max{0, min{A/, B{t - 1) 
+ r(t)-e(t-l)}}, 

Qit) ^ max{0, min{A', Q{t-l) 
+ a(t)-C(e(t-l))}}, 

e{t) < B{t) and C{e{t)) < Q{t). 

Here UD{C{e)) is a non-decreasing concave utility gained by 
transmitting C(e) bits. Since A < C(/Lt), we know that UdW 
is an upper bound on the achievable long-term utility with 
any energy management scheme. This statement can be proved 
using Jensen's inequality, following identical steps as the proof 
of Lemma [T] and we skip it to avoid repetition. 

B. An Asymptotically Optimal Energy Management Scheme 

Solution of Problem (|7]) jointly controls the data queue state 
and the battery state to avoid energy outage and data overflow 
while maximizing the utility. The main complexity in such an 
approach stems from the fact that the drifts of Q{t) and B{t) 
are dependent. In Fig. [3] we illustrate the connection between 
the service rate and the energy consumed at a time slot t for 
an AWGN channel with SNR dB and a{t) = 0, r(t) = 0. 
For instance, to provide 3 units of service, the node needs to 
consume ^18 units of energy. 

With this dependence, a critical factor one needs to take 
into consideration is the relative "size" of the data buffer with 
respect to the battery. In the sequel, we assume a large battery 
regime, where the time scale at which changes occur in B{t) is 
much larger than the time scale at which Q{t) varies. Namely, 
within the duration that some change occurs in B{t), Q{t) may 
fluctuate significantly. Technically, for an AWGN channel with 
an SNR 7, this assumption is equivalent to AI ^ ^{2'^ — 1)K, 
i.e., the total amount of energy in the battery is much larger 
than that required to serve a full data buffer worth of packets. 

Intuitively, in large battery regime, an energy control algo- 
rithm should give "priority" to adjusting the queue state to 
achieve a high performance. Consequently, it should choose 
e{t) such that the drift of Q{t) is always toward a desired 
queue state even though this may cause battery drift to be 
negative. Since battery size is large, such temporary negative 
drifts are expected to affect the battery discharge rate only 
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Fig. 4. Relation between (S'"', (5*''' and 5^'''. 



minimally. With these observations, we state the following 
theorem, which indeed verifies our intuitions. This theorem 
shows an asymptotic tradeoff between the achieved utility and 
the long-term rates of discharge and data loss as K oo. 
In this regime, the data buffer size K is large enough for 
the variations in a(t) to average out nicely over the time 
scale that Q(t) changes significantly. Consequently, we now 
define the long-term data loss rate as the data loss probability, 
i.e., pioss(-f^') — linv^oo 7 X^t^i -^k(^)' where the indicator 
variable = 1 if Q{t) = K and is identical to 

otherwise. 

Theorem 3. Consider any non-decreasing concave utility 
function Ud{-) such that ^ ^gff^'^'^^ < oo for all e > 0. 
For any A < C(/i), given any /3q > 2, there exists an 
energy management scheme Q that achieves a data loss prob- 
ability Puissi-^) = 0{K~^'^-^), battery discharge probability 
Pfischargei^'^) = (5(exp(-aQ Af )) for some ag > and 



under the large battery regime. 



Theorem [3] states that it is possible to have an exponential 
decay (with M) for the battery discharge probability and a 
polynomial decay (with K) for the data loss probability and 
at the same time achieve a time average utility that approaches 
the upper bound on the achievable long-term utility, Ud{^), as 
(logA')^/A'^. Note that Ui:,{\) can only be achieved with an 
infinite battery and data buffer sizes. We provide an outline for 
the proof, a full version of which can be found in Appendix ID] 
The proof for this theorem is also constructive as we first 
present Scheme Q, and then derive the performance metrics 
for this scheme. 

Consider the energy management scheme Q, where 



(r) 



Kir) 



Q{t) > K/2 
Q{t) < K/2 ' 



(8) 



and the drifts o\ and are chosen to satisfy the relationship 



ir)^ 



:]og£ 
' K ■ 



(9) 



From Fig. |4] we note that this choice of energy drifts corre- 
spond to a queue drift of \6i°-) \ = Pq^"^ '"g^ ' toward the state 
K/2, regardless of the queue state Q{t). The queue and battery 
drifts with scheme Q are illustrated in Fig. |5] We observe that 
even though Scheme Q regulates the data queue to a desired 
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State (i.e., K/2), the battery is always regulated towards full 
state (i.e., M). State equation for Q{t) is given by. 



Q{t + l) 

_ {min{K, Q{t) + a{t) - A - 
^ |max{0,Q(t) + a(t) - A + 5^'^)}, 

and the state equation for B{t) is given by, 

B{t + l) 

_ { {min{M, B{t) + r{i) - /i + 5^{'^}}+ 
^ \ {min{M, B{t) + r{t) - /i + 4''^}}+: 



Q{t) > K/2 
Qit) < K/2 ' 



(10) 



Q{t) > K/2 
Q{t) < K/2 ' 
(11) 



where {a}^ = max{0,a}. 

While p^^g{K) can be found using a method similar to 
the one used in Theorem [T] (this time for the data queue) to 
calculate discharge (^^)' calculation of pfseharge(^^) requires a 
careful analysis. More specifically, the battery drift in a partic- 
ular time slot depends on the data queue state, which makes 
the application of large deviation techniques for calculating 
?'dscharae(^'^) difficult. To show the dcsircd order results, we 
transform the problem in two steps as follows: 
(Tl) We construct a sequence of arrival rates such that A ^ 
C{fj,). In this limiting regime, from Eq. Q, s[''\s![^ i and 
hence 5^°-) I 0, for which we also use a sequence of K values 
that increase to oo. As a result, both the battery and the data 
queue will operate in the heavy traffic limit. Consequently, 
we apply the diffusion approximation to the joint queue and 
battery state process. The underlying continuous process is a 
2-dimensional refiected Brownian motion (RBM), where we 
label {(Q(t), B(t)), t > 0} as the diffusion approximation 
for the joint queue length and battery process. The reflections 
occur at the empty and full queue states, i.e., Q(t) = and K 
and at the empty and full battery states, i.e., B(t) ~ and M. 
Note that, the probabilities for hitting the boundaries calculated 
for flie associated RBM, P (Q(t) = K) and P (B(i) = 0) are 
identical to f {Q{t) = K) and P(S(t) = 0) respectively in 
the heavy traffic limit (Chapter 5 ifTSl ). Furthermore, since 
the heavy traffic limit poses a worst case for the probabilities 
under consideration, the order results of the form 0( ) shown 
in the heavy traffic limit hold for all A < C{^). 

As we show in Appendix iDl with Scheme Q, Q(t) is has 
a symmetric distribution around state Q(t) = K/2. Further, 
whenever it reaches Q(t) = K/2, it can go above or below 
Q(t) with equal probability. Consequently, we define Qu(i) — 
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\Q{t)- K/2\ +K/2, i.e., the RBM associated with the queue 
process in the upper half of the buffer Process Qu{t) has 
reflections at Q„(i) = K/2 and K, and one can see that the 
probabihty of loss P (Q(t) = K) = ip (Q„(t) = K). 
(T2) Closed-form analysis of the stationary distribution for the 
state of the RBMs is not possible except in some special cases 
given in |fT9l . Since our model does not fall in that category, 
the steady state reflection probabilities cannot be derived in 
closed form. To address this issue, we remove the upper and 
lower boundaries for the data buffer and the battery respec- 
tively, and allow {Q(t), t > 0} (and hence {Q„(t), t > 0}) 
to take on values in [1^/2,00) and {B(i), t > 0} to take 
on values in (— oo,A/]. Then, under scheme Q as given in 
®, we define pf,erflow(A') = ^ lim.^oo P(Qu(r) > K) 
and P?„derflow(^^) - ^(B(r) < 0). Using Theorems 1 and 
2 in 1201 . one can see that p^^^iK) = 0{K"^^) for some 
/3q > if and only if there exists some P'q > Q such that 
POTerflow(^) = 0(if"^«). Similarly, Theorem 1 and 2 in 1201 
imply that ?3£eharge(^) = 0(exp(-aQM)) if and only if 
Pu^,derflow(^'^) = 0(exp(-aQM)). Thus, it suffices to show 
the desired scaling laws for the aforementioned unbounded 
queue state and battery state processes. 

After transforming the problem as described above to one 
involving threshold crossing with Brownian motions, we de- 
rive the individual probabilities using the following techniques 
in Appendix |D1 First, we define a reward of one unit every 
time Quit) goes above K. Using renewal-reward theory, we 
find 

Po''veri,ow(^) = ^ ■ (12) 

By substituting 5^°-'> — P'q&I we have the desired 

scaling law for the queue overflow probability. Next, we 
define two power allocation policies that lead to a higher 
and lower battery discharge probabilities, respectively, for the 
same energy replenishment process. The steady-state proba- 
bility distribution of the battery state for these policies is an 
exponentially-distributed random variable. Applying the well- 
known squeeze theorem in the limit M — > oo, we obtain 
the desired scaling law for the battery discharge probability. 
Finally, proof for the convergence of the time average utility 
follows the same line of argument to that for Theorem [T] 

C. Exploring Tradeoffs Between Battery Discharge and Buffer 
Overflow Probabilities 

So far, we focused on achieving performance that was close 
to the optimal while keeping the probabilities of discharge 
and data loss low. In this section we will look at quantifying 
tradeoff between the probabilities of battery discharge and data 
loss. 

Theorem 4. For a sensor node with energy replenish- 
ment and a wireless channel with a rate-power func- 
tion C(-), there exists an energy management scheme £ 

that achieves Yivclm^oo jj'^ogPdischargei^^) = ^'"'^^ 

liniK-^oo^logpLW = _ ^(g(^-f' )-^) for any < 



Rate C{V) 




£?/i "power V 
Fig. 6. Relation between (5'"' and 5^^\ 

To prove this theorem (given in Appendix |E|, we consider 
a power allocation scheme £ that is given by, 

e^(t) =At-<^^''^ (13) 

for all t. The mean drifts for the battery state and the data 
queue state are given by J'^''^ and C(yU — J'-''^) — A, respectively. 
Applying the diffusion limits on these processes, we get the 
required probability results. 

Fig- m illustrates the relationship between the parameters of 
the system. Here, we note that j'"' = C(/i — 5'''^) — A. Any 
increase in S'^'^^ would lead to a corresponding decrease in 5''°-\ 
Since 5^^^ is proportional to the discharge probability decay 
exponent and (5'°' is proportional to the data loss probability 
decay exponent, we will get the given tradeoff. 

Theorem |4] shows that, while it is possible to achieve 
exponential decay rates for battery discharge and data loss 
probabilities, there is a tradeoff in the decay exponents. More 
specifically, by varying 5^^\ it is possible to increase (or 
decrease) the decay rate exponent for the data loss probability. 
However this will directly result a proportional decrease (or 
increase) in the decay rate exponent for the battery discharge 
probability. 

IV. Performance Evaluation 

Our theorems illustrate tradeoffs for energy management 
schemes in the buffer and battery size asymptotic regimes 
and showed optimality of some simple energy management 
schemes. In this section, we conduct simulations to evalu- 
ate the performance of those schemes in the presence of 
a finite battery and a finite data buffer. We construct the 
energy replenishment process r{t) using the real solar radi- 
ation measurements collected at the Solar Radiation Research 
Laboratory 11211 . The data set used is the global horizontal 
radiation or the total solar radiation using a Precision Spectral 
Pyranometer We use data from January 1999 to July 2010 
collected at 1 minute intervals. We assume that the energy re- 
plenishment process is proportional to the total solar radiation. 

In our simulations, we choose the battery size in the range 
of 10'^ — 10® mAh, as in lT2l . Fig. | 3ta)| shows a sample of the 
replenishment process r{t) over a 48 hour period. Also, we 
used the utility function [/(e) = log(l + 7e). 

To compare our scheme, we use the Throughput-Optimal 
(TO) policy given in Eq. (4) of |T5!|. The TO policy is given 

by, 

e™(i) = min{S(t),^-e}, (14) 
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where e is a constant such that — e) > A. 

A. Battery Constraints with Infinitely Backlogged Buffer 

In Fig. |7] we revisit the power allocation scheme B dis- 
cussed in Example [T] for an infinitely backlogged data buffer. 
The communication channel is AWGN with SNR 7 = 1, 
and we choose the polynomial decay exponent /3 ~ 2. From 
Theorems 12] and [T] we know that the TO policy should achieve 
an exponential decay for discharge probability compared to 
the quadratic decay for scheme B. On the other hand, the TO 
policy can not achieve the maximum utility while scheme B 
should achieve maximum utility as {log M)'^ / . Fig. | 3Ib)| 
plots the battery discharge probability as a function of the 
battery size M. As expected, the TO policy performs better 
than scheme B. However, the advantage of using policy B is 
evident in Fig. QJc)! which compares the time average utilities 
achieved by each scheme. It can be seen that, for the choice of 
parameters and data used in this simulation, scheme B achieves 
the maximum utility U{fi) for a battery size of 10^ mAh, 
whereas the TO scheme does not achieve the maximum utility 
even asymptotically. 

B. Buffer and Battery Constraints 

Fig. [8] compares the performance of power allocation 
schemes when both battery and buffer constraints are present. 
We simulate the data arrival process by generating a Markov- 
modulated Poisson process with mean A = 7.44 bits in every 
time slot. We use a two-state Markov chain to generate a 
bursty data arrival process. One state of the Markov chain 
generates a Poisson random variable with mean 25 bits and 
the other state generates a Poisson random variable with mean 
1. The mean of the energy replenishment process fj, = 9.58 
mAh per time slot and we choose /3q = 2. In Fig. | ^a)[ we 
fix the buffer size to 10^ bits and plot the battery discharge 
probability as a function of the battery size M. The discharge 
probabilities for both schemes should decay exponentially for 
both schemes. However, the decay exponent for scheme Q 
should be larger than the decay exponent for the TO scheme. 
In fact, the decay exponent for the scheme Q should be 
proportional to /i — C^^(A), whereas the decay exponent for 
the TO scheme is proportional to e < — C^^(A)) (given in 
Eq. ( fT4l i. As expected, the discharge probability for scheme 
Q decays faster than that of the TO scheme. In Fig. | ^b)[ we 
plot the data loss probability as a function of the buffer size 
while keeping the battery size fixed at 10^ mAh. We observe 
that the loss probability for the TO scheme decays faster than 
that for scheme Q. This trend is expected as the TO scheme 
should have an exponential decay compared to a quadratic 
decay for scheme Q. Fig. | ^c)| compares the convergence of 
the time average utilities to the maximum utility function for 
the two schemes. We observe that scheme Q converges to the 
maximum utility for moderately large buffer sizes 10^ bits). 
On the other hand, TO scheme does not achieve the optimal 
utility even asymptotically. 

Fig. |9] compares the performance of power allocation 
schemes with increasing traffic intensity. We define traffic 
intensity as p = = — r-. We fix the buffer length 



at 10^ bits and battery capacity is set at 10^ mAh. In Fig. I 21a)| 
we observe that the discharge probability increases with traffic 
intensity. For values of p < 0.97, scheme Q performs almost 
an order of magnitude better than the TO scheme in terms 
of the discharge probability. For traffic intensities close to 
unity the scheme Q degenerates to the TO scheme and their 
performances converge. Fig. | 2tb)| shows that the data loss 
probabilities for both schemes also increases with increasing 
traffic intensity. Similar to the discharge probability, for values 
of p < 0.97, the loss probability for scheme Q is almost an 
order of magnitude lower than that for the TO scheme. This 
can be explained by the higher discharge probability for the TO 
scheme leading to severe performance degradation. Finally, we 
observe in Fig.| ^c)| that the time average utility decreases with 
increasing p. This is a direct consequence of increasing battery 
discharge and data loss probabilities leading to sub-optimal 
performance in both schemes. As p — > 1, the performances of 
the two energy management schemes degrade highly. 

C. Trade-offs Between Buffer Overflow and Battery Discharge 
Probabilities 

In Fig. [Tol we numerically evaluate the trade-off between 
battery discharge and data loss given in Theorem ID We 
use the data arrival and energy replenishment process used 
previously. Fig |10(a)| illustrates that in order to increase 
the decay exponent for the battery discharge probability, the 
energy management scheme has to decrease the exponent for 
the data loss probability. We choose three operating points on 
this curve and evaluate the battery discharge and data loss 
scaling for these points. As we go from operating point 1 
to 3, the data loss probability decay exponent increases and 
the battery discharge probability decay exponent decreases. In 
Fig. |10(b)[ we observe that the quickest decay for the loss 
probability is for operating point 1. In Fig. |10(c)l as expected, 
we see the opposite effect wherein the discharge probabiUty 
decays fastest for the operating point 3. 

V. Conclusions 

In this paper, we studied the basic limits and associated 
tradeoffs for energy management schemes in energy replenish- 
ing sensor networks. We showed that it is possible to observe a 
polynomial decay for the discharge probability with increased 
battery size, and at the same time achieve 8((log A/)^/A/^) 
convergence to the maximum achievable utility using a simple 
energy management scheme. We showed the strength of this 
result by showing that it is not possible to simultaneously 
observe an exponential decay for the discharge probability 
and achieve maximum utility. With the insights drawn, we 
addressed the problem of energy management with buffer and 
battery constraints. We showed that, in addition to achieving 
Q{{\ogK)^ / K^) convergence to the optimum utility, it is 
possible to achieve a polynomial decay for the data loss 
probability and exponential decay for the the battery discharge 
probability using a simple energy management scheme. 

To analyze the buffer and battery processes we made use 
of large deviations theory and diffusion approximations. The 
main advantage of using these tools in our work is that it 
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allows analytical tractability while keeping the system model 
fairly general in nature. Finally, we numerically illustrated the 
performance of the our simple energy management schemes 
along with that of another existing scheme, and demonstrated 
that our scheme can perform up to an order of magnitude better 
in terms of outage probabilities while achieving the maximum 
utility asymptotically. 

One possible future direction of research is the design of op- 
timal or near-optimal practical energy management solutions 
in the presence of channel fading and multi-user interference 
and to build distributed algorithms to realize these schemes. 
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Appendix A 
Proof of Lemma[T] 

To prove this lemma, we first use the finite form of Jensen's 
inequality to establish, 

t=i \ t=i J 

Since this inequality holds for any finite r, passing the limit 
r oo, the inequality is preserved, 

OO / OO \ 

lim - y U{e^{t)) < lim C/ I i y e^{t) ] 

= [/( lim iye'5(i)) , (15) 

\ r— >-oo T — I 
\ t=l / 

where (fTSl l follows since [/(•) is a continuous function ll22ll . 
From conservation of energy, we have 

lim - y^e^{t)< lim - J^ r{t) ^ ^i, (16) 

r— foo T — ^ r— >oo T — ^ 

t=l t=l 

since M < oo. Combining Eqs. ( fTSl l and ( fTSl ). we have the 
required result, 

-j^ OO 

lim - y U{e^{t)) ^U^ < U{fi). (17) 

r— foo T — ^ 

t=l 

Appendix B 
Proof of Theorem[T] 

In this appendix, we prove that the power allocation scheme 
B achieves the scaling properties given in Theorem [TJ First, 
consider a general form of Scheme B: 



fi-5-, B{t)<M/2 
fi + 5+, B{t)>M/2 



(18) 



for some pair (5^, 5^ , that will be chosen later. We will show 
that the desired solution involves 5~ =5^ = 5^ . 

Depending on whether the battery state is less than (or more 
than) half full, the expected drift of the battery state becomes 
positive (or negative). Given B{t) < M/2, the asymptotic 
semi-invariant log-moment generating function of the battery 
state drift, d^{t) = r{t) - {p- - S^), is 



Arf-(s) = lim - logE 



exp 



:J2d-{t) 



= Ar{s) — s(/i — S ). 



(19) 
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Where Ar{s) is given by Eq. ([3j. Let s*^_ be the negative rooj^ 
of Ad-(s),i.e., Arf-(s*_) Ar(s*_)-s*_(^-J") = 0. Also 
as S~ — )• 0, s*_ — !• 0. 

Before we prove Theorem[T] we state and prove the follow- 
ing lemmas. Lemma|2]gives the rate of decay of the probability 
of battery discharge with respect to the battery size M for the 
Scheme B. Lemma [3] expresses the rate decay exponent .s^_ 
for scheme B in terms of the asymptotic variance of energy 
replenishment process r{t). 

Lemma 2. The probability of battery discharge under 
Scheme B with battery size M follows P^jschan>ei^^) — 
^exp where is the negative root of fi^-{s). 

Proof: Fix a constant A> Q and decompose the time line 
into intervals, such that each interval is of length [^] and 
the ith interval ends at time slot ti = Assume that the 

system has been active since t = — oo. We define Ei as the 
event that the battery is empty at the end of time slot and the 
last time the battery was half full (i.e., M/2) is some instant 
during the interval -i = [-(j + l)r|jl + '^^-A^V\- The 
event of an empty battery at time slot can be decomposed 
as a union of events Ei, 



-Fdischai" 



(20) 



A necessary condition for event Ei to occur is. 



t=-(*+i)riii+i 

Using Chernoff 's bound, for any 9i > 0, 



M 

Y 



(21) 



vt=-(»+i)riii+i 



< E 



= E 



*="(''+i)r^ii+i 



cxp -0, 



M 



cxp 



X exp Oi{i + 1) 




{^l-5 ) ) exp ( -Qi^^ 



/ M 

exp 



1 1-^(^-5-)) -^A.MO 



A 



A 



where ei{M, 0i) ^ as M oo. 



(22) 



''Note that (0) 
will exist. 



and 



> 0. Consequently 



< 




Fig. 11. A geometric proof for the existence of J and <5 > such that for 
every i > J, fi(d) > -y + iS. 



In order to find the tightest bound for each i, we choose 
6* > to maximize, 

MO) = 0(1- ^(/i - S-)^ ^A,.(-0), (23) 

over all 6* > and let 7 = inf j>o supg>o /i(6') = 
mii>ofi{0*). We can rewrite fi{6) as, 

. - _ _ ^ Am - s-)e + A.(-^) 



A 



A 



A 



Since limT-_>.oo 7 J2t=i ^ [''(^)] = A' > M ^ ' the function 
{fi — d~)9 + Ar{—9) has a negative slope at ^? = 0. Hence, we 
can choose some 6 > 0, such that (/i — 6^)9 + Ar{—9) < 0. 
This implies that there exists a J and a /3 > such that for 
every i > J, 



as illustrated in Fig. [TT] Returning to Eq. 

00 



(24) 



4=0 





M 



J,i9) + e,{M,9) 

^ f M ' 

E exp(-y 



E ^-p(-Y 

i=J+l ^ 



i=0 



^ + 9*) 
0<t<J 



M 



j + i/3+ inf ei{M, 

i>,J 



cxp 1 — -■y 



(J + 1) cxp min e(Af, 9*) 

' 0<i<J 



exp 



(-f ((J + l)/3 + inf,>je,(A/,0~)) 
1 - exp {-P^) 



(25) 



As M — > 00, 



limSUp — l0gPdi,,j,arge(^) < "7 



Since this inequality holds for any A > 0, we let A cx) as 
follows: 

2 

limSUp — l0gpfi,eha,-ge(A^) 



From Eqs. ( |26] l and dSTT i we have, 



J™ T7logPdischai-ge(^^) 
A/— foo iVjf 



M— i-oo 



M 



< — inf sup 



inf T sup 

T>0 9>o 



fi-5--] -Ar{-e) 



■ inf sup [e (1 - T{^i - S)) - TAri-e)] 



inf Tsup 



T>0 



9>0 



(26) 



Next, we find the lower bound. For some T > 0, a sufficient 
condition for the battery to be empty at some time slot in the 
interval [-[TM/2],0] is that, 



^ (e^(t)-r(t)) >M. (27) 

We can lower bound pfischaroe(^^) using the union bound, 

'TM 



P B{t) = within some t e 



2 





,0 



U sw = o < E 

it=-rTJ\//2] / t=-rTA//2] 



PLha.,e(^^)- 



'iB{t)=0) 

(28) 



We also have 



This gives us pi,i,a>-ge(^^) = © (cxp ^)). 
Lemma 3. The asymptotic variance of r(t), a 
limT_j.oo yvar ''(O) satisfies 



dS- 



Proof: First, we define Aj^j(O) 



s=0 



Taylor series expansion of A^- (sj^_ ) about s = gives 

o^A,-isn = Y.^t\o)^-^ 



n=0 



A^+aW(o).*_ +a(^J(0)^ 

=0 

^A(")(0)^-(^-<5-K- 

ri = l 

A.s*_+Ea(")(0)^^-(a.-5)s 



We define, Z^/r 
Consequently, 





E (/.-<^--r(t))> 



M 



TM 



r{t)) > Y 



(29) 
r{k)). 
1 



E (M-^ 

Now, limj,/^oo E [Za/.t] = -(5" < < ^ for all T > 
Applying the Gartner-Ellis Theorem, we get, 

lim ^logF ( Zm,t > ^ 

M->-oo M \ 1 



— sup 
—Tsup 

s>0 

—Tsup 



is-.(/i-<5) + TA,,(-|; 



TV T 



1 



A^ 



-e[tJi-5--]-Ar{-i 



(30) 



Combining Eqs. ( l28T l and ( l30l l, we have, 
2 „ 



> — inf T sup 



(31) 



Rearranging the terms, we have 

°° ( „* \n-l 

EA(")(0)if^ = -J- 

n=2 

Differentiating with respect to 5~ , we have. 



As 5- 



0, s* 



= -1. 



the above expression reduces to. 



85 

Since Af^^(O) = a^, Eq 



<5-=0 



A(^)(0)i = -1. 



becomes. 



d5- 



(5-=0 



i-2 ■ 



Lemma [3] implies 



9s* 



and hence. 



0(5- 



o(r 



where o{6 )/d — > as (5 — > 0. 

Substituting this in Eq. ( |32] |. we have. 



Pdischaree(^^) = O CXp 



o(r 



12 



By choosing S = a and a ~ fSa"^ we have 

Next we show that with (5+ = a ^°ff^ the scheme achieves 

an average utility U'^ such that U{n) - [7^ = 9 (^^^ 

The instantaneous utiUty U{e^{t)) is zero with an 0{M^^) 
probability. For the remaining time, the utility alternates be- 
tween and as illustrated in Fig. [J] The Taylor series 
expansion of the utility functions about /i will be. 



U+ = C/(m) + U^'\ti)6+ + U^^\^i){5+f + o((<5+)2), 
and, 

U- = U{ii) - U^^\p)5- + U^^\^i){5-f + o{{5~Y). 

We define p+ as the fraction of time that B{t) > M/2 and 
= 1 - p+ as the fraction of time that B{t) < M/2. The 
average utility can be written as. 



= u{^l) + u^^\^i)ip+s+ - p-s-) + e 



(logM)^ 



(37) 

log M 
M 



where Eq. ( |37] | follows from the fact that (5 , (5+ = a 
and Pd1scharge(^0 = 0(A/'^*) where /3 > 2. 

From conservation of energy, the replenishment energy is 
consumed completely except for the amount lost due to battery 
overflows. Thus, 

p+(^^ + S+) +p-{l- pLharge(A^))(A* - S-) 

= p(l-pLrflow(^^)), (38) 

where Poveiflowl^^) '^^e probability of the battery being full 
under the power allocation scheme B. By a trivial extension 
of Lemmas |2] and |3] it can be shown that p^ygj.g^^(A/) = 
8 (AZ-^*). We can simpHfy Eq. ^ as. 



P+6+ -p-s- = {p-{p- 6-)- fi)e{M-'^) 

= e (M-'^) . (39) 



By substituting Eq. ( [39] l in the first-order term of Eq. ( |37] |. 
we observe that the scheme B achieves U{p) — = 
e((l2SJ^). Choosing 5« = 5+ 
completes the proof of Theorem [T] 



al2Sj^ in Eq. ® 



Note that s*^ > exist^ if and only if E [d^{t)] < 0. 
Therefore, for scharge(-^^) ^'^ decay exponentially with M, 
we require, 



[£•5(0] <E[r(t)] 



(40) 



On the other hand, if E [(i'5(t)] > 0, there exists no rate s > 
at which the battery discharge probability decays exponentially 
with M, i.e., pfi,ehai-„e(^j^) = f^(exp(-sA/)) for all s > 0. By 



substituting ac 



in Lemma m we get the required scaling 



law pLharge(^^) = e(cxp(-a,M)). 

The difference between the utilities is given by, 

U{fi)-U^ = U{^,)~ lim i^C/(e'5(t)) 



(a) 

> ?7(p) - U 



1 



lim -Ve'5(i) 

r^oo T ^- — ^ 
^ t=l y 



(&) 



(41) 



Where (a) follows from Eq. ( fTSl ). (6) follows from the ergod- 
icity of e'^it) and (c) follows from the fact that p>E [e^{t)] 
and U{-) is an increasing function. This completes the proof 
for Theorem 121 

Appendix D 
Proof of Theorem[3] 



Appendix C 
Proof of Theorem[2] 

Consider any ergodic energy management scheme S that 
uses (t) units of energy in the time slot t. Note that scheme 
S can be deterministic or randomized. The asymptotic semi- 
invariant log moment generating function of the net battery 
drift = e^{t) — r{t) is given by Aj^s{s). First, we state a 
lemma that gives the discharge probability scaling for scheme 
S. Since this lemma is a minor modification of Lemma |2l we 
omit the proof in this paper We direct the reader to ll23l for 
the detailed proof of this lemma. 

Lemma 4. The probability of battery discharge under 
Scheme S with battery size M follows P^schargei^) ~ 

0(exp(— sjljs Af )), where s'^s is the positive root of flj^s{s). the resulting analysis will not change. 



As discussed in Section IIII-BI our proof is constructive. 
We use the energy management scheme Q given in Eq. ([8]). 
Also, in Step (Tl) we show in the heavy traffic limit that, the 
data loss and the battery discharge probabilities can be found 
using the appropriate reflection probabilities of the associated 
2-dimensional RBM. Further, we showed in Step (T2) that 
the scaling laws for the data loss and the battery discharge 
probabilities are preserved when they are approximated by, 
respectively, the associated overflow and the underflow prob- 
abilities of the Brownian motion without reflections. We find 
the individual probabilities in the following lemmas. 

Lemma 5. For energy management scheme Q, given any 

Proof: If we assume the starting state of the Brow- 
nian motion to be Q(0) = K/2. Note that due to the 
strong Markovian property of a Brownian motion |>24| . the 
instants {r„, n = 1,2,...} at which the system re- 
turns to state K/2 (i.e., Q{Ti) = K/2) is probabilistically 
equal to the starting state. Hence we can study these re- 
newal epoch^ to obtain steady state properties for the data 
queue process. Now, consider the random variables r„ = 
argmint>o{QW = A72,Q(0) = A72,Q(0+) > K/2} 
and Tl ^ argmint>o{Q(i) = K/2,Q{0) = A72,Q(0+) < 



da 



'Since A^s(O) and 
will exist. 

'if we assume the starting state to be Q{0) 7^ 0, we can simply consider 
the process to be a delayed renewal process. The steady state properties in 



13 



K/2]. The hitting time distributions can be calculated as Il24l . 

K 



Tu>t + T 



Q(r) = 



= $ 



2 
exp 



= P T, > f 



K 



Q(r) = y-e 



(42) 



where ^ /^^ exp (^-^) dx. Since Eq. ^ holds 
for all T > and e > 0, the random variables T„ and 
Ti will have the same distribution. Furthermore, once the 
process Q{t) = K/2, it can go above or below K/2 with 
equal probability. Consequently, we need to study the renewals 
associated with Q„(t) = \Q{t) - K/2\ + K/2 and can 
find ¥{Q{t) > K) = iP(Q„(0 > K), which is identical to 

^overflow 

If we define a unit reward (i.e., R{t) = 1) for every time t 
that the process Q„ [t) > K then. 



lim P(Q„(i) > K) = lim E [R[t)] 



we can write, 

Rn] 



(43) 



From renewal-reward theory 

where E [i?„] is the expected award accumulated in one 
renewal period, and E [X] is the expected length of the renewal 
period. To get the correct expression for limt^oo E we 
need to write the expressions for E [Rn] and E \X] carefully. 
We define E [^(e)] as the expected time for process Q(i) to 
return to K/2 given that it starts at K/2 + e. The expression 
for E [X{e)] is given by l26l, 

^[^(^)] = ^- (45) 

Similarily, we define E [i?„(e)] as the probability of reaching 
K before K/2 starting at K/2 + e. Passing the limit e J, will 
give the expected reward accumulated in one renewal period. 
Applying the expression for this probability from 



;[i?„(e)] = 



exp 



- 1 



exp 



o{e) 



exp 



2<5(") K 



1 



Dividing Eqs. ( |46] | by ( l45T l and passing the limit e | 0, we 
have. 



lim E [R{t)\ = lim 

t— >-oo e^O 



-p(2ll^f)-l 



(47) 



Evaluating the limit e | 0, and noting that the stochastic 
process will be in the upper half of the buffer with probability 
1/2, for large K we have, 

\2 



overflow 

{K) 



Si-) 



■ exp 



By choosing S'^''^ = /?' ct^I^^, we have. 



Poverflowl-'^ J — PQ '^a ^ 



o(k-^''^ 



exp (-^Q log A'^ 



(48) 



Lemma 6. For f/ie energy management scheme Q, 
liniM^ooi^log(p«,,,^„jAf)) = -Hiii^lpl), ^here 
C^^(-) /i f/ie ;«verie of the rate-power function C(-). 

Proo/.- To calculate pSderflow(^^) = 

P(B(t) < 

0), first we define power allocation policies Qi, 



(r) 



and Q2, 



(49) 



(50) 



Let the battery underflow probabilities associated with 
schemes Qi and Q2 hepf^^^^^jM) andp^^^^^^^^{M), respec- 
tively. For the same energy replenishment process {r{t),t > 
0}, the net battery drift is defined as d'^(i) = r{t) - e^{t), for 
each scheme S E {Q, Qi, Qz}- The net drifts will have the 
following relation. 



d^^t)>d^{t)>d^'{t) yt. 



It follows that. 



Punderflow(^-^) < 



underflow 



(A/)<pSLflow(A^) 



(51) 



(52) 



The Brownian approximation for the battery process under 
policies Qi and Q2 is an exponential random variable, and 
the undeflow probabilities are given by ITSl . ll27l . 



(r) 



P?nderflow(^^) = CXp | - ^ 



M 



and. 



pSderflow(A^) = exp 



2S. 



(r) 



-M 



(53) 



(54) 



(46) Substituting Eqs. ([53]l and ^ in Eq. (I52]i we have. 



26. 



(r) 



exp 



-^M <P?„derflow(*0<CXp 



2(5 



(0 N 



(55) 



For a rate-power function C(-), we havejj"^ =/i-C- 1(A) + 
e(5('^)) and 4''^ = M - (^"H^) + e((5(°)). With (5(") = 
/jqtj^ '°|,^ — > as K — > 0, we can apply the squeeze theorem 
in Eq. i55[ to get the required result. 

■ 

To illustrate Lemma |6] we consider the AWGN chan- 



(r) 



nel capacity given in Eq. We have, 6 

exp((A+^<°))log2)-l ^(,) ^ cxp((A-g(°))log2)-l 



We 



14 



can use the power series expansion of the exponential function 
to get, 

1 _ (A + 5(''))2(log2)2 



and. 



- (A + <5(°^)log2 + 



7 

— exp(Alog2) 

7 



(56) 



g2 



(A-5("))2(log2) 



— exp(Alog2) 

7 



Eq. dSSll, 



(57) 
we get 



Substituting these expressions 
Pdlha>-ge(A^) = e(exp(-aQA/)). 

Finally, we show that Scheme Q achieves an average 
utility U§ such that Ud{\) - U§ = 9 The 
instantaneous utility will be zero when the queue is empty 
or when the battery is discharged. Due to symmetry of the 
Brownian approximation, the empty buffer probability will be 
the same as the data loss (i.e., full buffer) probability. Since 
-PdischarEe(-^) = 0(exp(— M)), Under the large battery regime 
we can ignore the discharge term. The average utility U§ can 
be written as, 

Ug^l {UoiX + S^""^) + UoiX - <5("))) (1 - (AO) 

(58) 

(59) 



UD{x) + e 



where Eq. ( |58] l follows from the fact that P^^^iK) = 
0( A'^'^Q ) for some /3q > 2 and Eq. (|59] l comes from choosing 



^2 log K 



This completes the proof of Theorem [3] 



Similarily, by applying the diffusion limit to the data buffer 
process we can find the buffer overflow probability as, 

pi / 25(''' 

pLmoJK) = lim :,^{Quit) > K) = cxp -^K 
t^oo 2 \ (^i 

(62) 

where = C{n ~ J*'')) - A. Substituting this value of (J*"' 
in Eq. ( |62] l. we get the desired result. 
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Appendix E 
Proof for Theorem|4] 

To prove this theorem, first consider a power allocation 
scheme 8 that is given by. 



(60) 



We define the battery drift as d^attei-ylO - ^(*) - The 
mean drift for the battery process is equal to iJ^'"'. Applying 
the diffusion limit, we can write the discharge probability for 
this energy management scheme as ifTsl . ll27l . 



pfnde.flow(^^) = lim P (B(t) < 0) = exp - 



-M 



(61) 
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